
April 2017 



Establishing and sustaining 
networked improvement 
communities: Lessons from 
Michigan and Minnesota 



Amy R. Proger 
Monica P. Bhatt 
Victoria Cirks 
Deb Gurke 

American Institutes for Research 


Summary 



A networked improvement community is a collaborative research partnership that uses 
the principles of improvement science within a group of organizations to learn from 
promising practices developed in each context and how they may be adapted to other 
contexts. Regional Educational Laboratory Midwest worked with educators in Michigan 
and Minnesota to establish two networked improvement communities during the 
2015/16 school year. The collaborations revealed that the following tasks are important 
in establishing successful networked improvement communities: 

Building a cohesive team with participants representing different types of expertise. 
Reducing uncertainty by clarifying what participation entails. 

Building engagement by aligning work with ongoing efforts. 

Using tools and resources from improvement science to identify a problem that is 
important and specific enough to be able to act on. 

Embedding capacity building to develop additional expertise for using continuous 
improvement research to address problems of practice. 
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Summary 


There is growing interest in the ability of improvement science — the systematic study of 
improvement strategies to identify promising practices for addressing issues in complex 
systems (Improvement Science Research Network, 2016) — to spur innovation and address 
complex problems. In education this methodology is often implemented through collabora- 
tive research partnerships in which researchers and practitioners work together to system- 
atically test and refine theories of change in real-world settings. A networked improvement 
community is a collaborative research partnership that uses the principles of improvement 
science within networks of organizations to learn from varied implementation of new ideas 
across contexts. While the central work of a networked improvement community is to 
identify a specific and actionable problem and collectively address it through an iterative 
process of designing, implementing, testing, and redesigning promising new practices, the 
learning from these iterative cycles can be brought back and applied to the local contexts 
of the networked improvement community participants (such as classrooms, districts, and 
states), potentially affecting education practices more widely. 

Although there is practical guidance for how networked improvement communities 
should structure this work, few published accounts describe the process of forming a net- 
worked improvement community. This report describes the process of forming networked 
improvement communities in Michigan and Minnesota after state education agency 
leaders requested assistance from Regional Educational Laboratory (REL) Midwest to 
support state-led efforts to use improvement science to raise student achievement and 
narrow achievement gaps in schools with the widest achievement gaps (focus schools). The 
resulting collaborations led to the establishment of two networked improvement commu- 
nities during the 2015/16 school year, one in Michigan and one in Minnesota, focused on 
improvement in schools identified as needing support under their accountability systems. 
The REL Midwest project team used guidance from the literature and other improve- 
ment science efforts (for example, Bryk, Gomez, Grunow, & LeMahieu, 2015) to direct its 
activities. 

Each networked improvement community has a slightly different history and emphasis. 
The Michigan Focus Networked Improvement Community works across five focus schools 
— schools with the largest achievement gaps — in two districts to address disparities in 
student achievement within schools. The two districts are each part of an intermediate 
school district, a regional education service agency that provides consolidated support 
services to districts in an assigned service area and thereby plays an important role in pro- 
viding professional development and supporting pilot programs in districts. Participants in 
the Michigan Focus Networked Improvement Community include state education agency 
representatives, intermediate school district administrators, district representatives, and 
focus school principals. 

The Minnesota Statewide System of Support Networked Improvement Community seeks 
to improve state supports to six Regional Centers of Excellence that serve focus schools. 
In Minnesota, the Cross-agency Implementation Team oversees the implementation of the 
statewide system of support. Its members include leadership and content specialists from 
both the Minnesota Department of Education and the Regional Centers of Excellence; 
they also serve as participants in the networked improvement community. 


The goal of establishing both networked improvement communities was twofold: to expose 
the state education agencies to a process that could be used to scale initiatives and to 
engage agencies at a level that would leave them able to use the process with other initia- 
tives. Networked improvement community participants are now focusing on sustainability, 
using what they learned in the first year as the foundation for maintaining key processes 
and functions. This report aims to guide other researchers, state education agency leaders, 
and district leaders as they establish networked improvement communities in different 
contexts. 

Throughout the process of establishing networked improvement communities in Mich- 
igan and Minnesota, the REL Midwest project team met regularly with colleagues with 
expertise in continuous improvement research to discuss the progress of each project 
and any challenges encountered. These conversations, in addition to feedback provided 
by networked improvement community participants during project debrief conversations, 
revealed that the following tasks are important in establishing successful networked 
improvement communities: 

• Building a cohesive team with participants representing different types of expertise. 

• Reducing uncertainty by clarifying what participation entails. 

• Building engagement by aligning work with ongoing efforts. 

• Using tools and resources from improvement science to identify a problem that is 
important and specific enough to be able to act on. 

• Embedding capacity building to develop additional expertise for using continuous 
improvement research to address problems of practice. 
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Why this project? 


There is growing interest in the ability of improvement science to spur innovation and 
address complex problems of practice. Improvement science can be traced to efforts to 
improve industrial manufacturing practice, but the process of improving practice through 
systematic inquiry has garnered interest in other disciplines as well, including education 
(Bryk et al, 2015; Lewis, 2015; Sparks, 2013). Educators have long engaged in strategies 
to improve practice, but because of their singular focus on their own practice, their find- 
ings are seldom shared beyond their school, district, or state (Zeichner, 2001). As a result, 
promising practices often are not implemented in new contexts or are implemented on a 
large scale without the necessary capacity to do so and without careful attention to the 
challenges of implementation (Bryk et al., 2015; Coburn & Stein, 2010). For example, a 
study of the failure of a large-scale initiative funded by the Gates Foundation to convert 
large high schools into smaller, more personalized learning settings recommends “learning 
by doing” on a small scale before moving to large-scale implementation (Bryk et al., 2015). 

In education, improvement science is often implemented through collaborative research 
partnerships in which researchers and practitioners work together to systematically test 
and refine theories of change in real-world settings. Networked improvement communities 
are one such partnership. 

Although there is practical guidance for how networked improvement communities should 
structure this work, few published accounts describe the process of forming a networked 
improvement community. The Carnegie Foundation for the Advancement of Teaching pro- 
vides guidance for researchers and educators who intend to form networked improvement 
communities. Examples used in the guidance include forming networked improvement com- 
munities around increasing community college graduation rates and improving support for 
novice teachers (Bryk et al, 2015). However, the guidance focuses on conducting continu- 
ous improvement research within the networked improvement community after it has been 
established rather than on establishing a networked improvement community (Russell et al., 
in press). The literature addressing the social aspects of establishing a networked improve- 
ment community is also limited, including exploring the dynamics of the participants and 
how they negotiate their roles and responsibilities (Patton, 2010). Although connections can 
be made to how other collaborative partnerships, such as communities of practice or profes- 
sional learning communities, are established, there is a lack of information about the process 
of leading, organizing, and operating networked improvement communities. 
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This report addresses these gaps by describing the process of forming networked improve- 
ment communities in Michigan and Minnesota and the lessons learned by Regional Edu- 
cational Laboratory (REL) Midwest in the process. It aims to guide other researchers, state 
education agency leaders, and district leaders as they establish networked improvement 
communities in different contexts. 


What is a networked improvement community? 

In a networked improvement community educators working in different contexts use iter- 
ative cycles to simultaneously design, implement, test, and redesign promising practices on 
a small scale (Bryk et al, 2015). By starting small, educators gain a deep understanding of 
a problem and the complex education system in which it operates. They are able to design 
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solutions that address the problem, while taking into account the local context and the 
challenges it presents; as a result, the solution is more likely to be successful (Bryk et al, 
2015). As the networked improvement community experiences success, participants can 
gradually move from implementing small changes in practice in one or two sites to larger 
changes in practice across several sites. 

Networked improvement communities are a relatively new type of collaborative research 
partnership between researchers and educators, popularized in education settings by the 
Carnegie Foundation for the Advancement of Teaching (Coburn, Penuel, & Geil, 2013). 
A networked improvement community is similar to other quality improvement approaches 
(such as design-based implementation research and rapid-cycle evaluation) that focus on 
learning from efforts to improve the design of programs in local contexts so that they can 
be implemented in different contexts. 

A general scientific research process includes eight stages: research aims (identifying the 
research questions), literature review, research design, instrumentation, pilot testing, 
data collection, data analysis, and a research report (Postlethwaite, 2005). In networked 
improvement communities and similar quality improvement approaches, participants use 
a collaborative process to identify and select the research question and implement the 
research process in an iterative manner. With facilitation from researchers, educators idem 
tify problems of practice, the factors that drive those problems, and promising solutions to 
those problems. They then engage in continuous improvement research, which involves 
iterative cycles of designing, implementing, testing, and redesigning solutions, while learn- 
ing from variation across the settings that form the networked improvement community 
(Langley et al., 2009). 

Networked improvement community participants can include educators working in a range 
of capacities and at different levels of the education system (such as the school, district, or 
state level). One example is the Building a Teacher Effectiveness Network (Bryk et al., 
2015), which is facilitated by the Carnegie Foundation for the Advancement of Teach- 
ing, the Institute for Healthcare Improvement, and the American Federation of Teachers. 
Participants include principals and teachers from the Austin Independent School Dis- 
trict, Baltimore City Schools, and the charter network New Visions for Public Schools. 
Another example is the Mathematics Teacher Education Partnership (Martin & Gobstein, 
2015), which is facilitated by the Association of Public and Land-Grant Universities and 
an unnamed member university. Participants include leaders from more than 90 member 
universities and more than 100 K-12 school districts. 

Networked improvement communities are characterized by four features (Bryk et al., 2015): 
they are focused on a well-specified, common aim; they are guided by a shared working 
theory of the education system in which they operate and how it can be improved; they 
use improvement science methods and measures to spur improvement in testable itera- 
tions, such as rapid plan-do-study-act cycles (see box 1 for an explanation of these cycles 
and other key terms); and they are organized to share and integrate the practices and pro- 
cesses that they develop in other contexts. 
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Box 1. Key terms 


Collaborative research partnership. A long-term collaboration between practitioners and 
researchers that investigates problems (Coburn et al. , 2013). Types of collaborative research 
partnerships include research alliances, design research partnerships, and networked improve- 
ment communities. Collaborative research partnerships are also referred to as research- 
practice partnerships. 

Continuous improvement research. An approach to research that is based on improvement 
science principles and involves implementing, monitoring, and testing practices in iterative 
cycles (Langley et al., 2009). Continuous improvement research is also referred to as a model 
for improvement. 

Fishbone diagram. A visual representation of a problem and its root causes. At the “head” of 
the fish is a problem statement. Each major "bone” represents hypothesized root causes of 
the specified problem. Through discussion, networked improvement community participants 
flesh out the “bones” by listing the factors that contribute to the root causes (Bryk et al., 

2015) . 

Focus school. A school that has one of the largest achievement gaps in a state. 

Implementation science. The study of the factors that influence the effective use of evidence- 
based interventions in practice (Fixsen, Naoom, Blase, Friedman, & Wallace, 2005). 

Improvement science. The systematic study of improvement strategies to identify promising 
practices for addressing issues in complex systems (Improvement Science Research Network, 

2016) . 

Intermediate school district. A government agency that assists local school districts in provid- 
ing programs and services in Michigan. It often operates at the county level but can include 
multiple counties. Intermediate school districts are sometimes referred to as regional educa- 
tion service agencies or education service agencies. 

Networked improvement community. A collaborative research partnership that combines the 
principles of improvement science within networks of organizations to learn from varied imple- 
mentation of new ideas across contexts. Participants in a networked improvement communi- 
ty identify a problem that is common across contexts and engage in rapid cycles of design, 
implementation, testing, and redesign to develop solutions (Coburn et al., 2013). The term was 
introduced in in 1992 by Douglas Engelbart, an engineer who saw the promise of using collec- 
tive action to improve the functioning of systems (Engelbart, 1992). 

Plan-do-study-act cycle. An iterative process through which an innovation is designed, imple- 
mented, tested, and redesigned (Deming, 1993). In recent years it has been applied to health- 
care and education as a method for continuous improvement by testing and refining changes 
on a small scale (Bryk et al., 2015). 

Priority school. One of the most persistently low-performing schools in a state. 

Professional learning community. A group of educators working together to improve practice 
through the examination of student data (Fiord, 2009). 

Regional Centers of Excellence. Six centers established by the Minnesota Department of Edu- 
cation as part of its request to receive an Elementary and Secondary Education Act waiver. 

(continued) 
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Box 1. Key terms ( continued ) 


The centers implement Minnesota’s statewide system of support, which offers differentiated 
recognition, accountability, and support for priority schools and focus schools. 

Root-cause analysis. A process through which participants in a networked improvement com- 
munity define the problem that will be the focus of their efforts. Root-cause analysis moves 
participants from a broad problem to one that is specific enough to be able to act on. Root- 
cause analysis is also referred to as causal systems analysis. 

Statewide system of support. A framework for providing differentiated supports to low-performing 
schools. Each state that received an Elementary and Secondary Education Act waiver was 
required to implement a statewide system of support that focused on priority schools and focus 
schools. The U.S. Department of Education granted wavier requests to the Michigan Depart- 
ment of Education in 2011 and the Minnesota Department of Education in 2012. 

Systems. “Organization(s) characterized by a set of interactions among the people who work 
there, the tools and materials they have at their disposal, and the processes through which 
these people and resources join together to accomplish its work” (Carnegie Foundation for the 
Advancement of Teaching, 2015). Examples include a classroom, school, district, state, or 
network of two-year or four-year colleges. 

Theories of practice improvement. A description of the link between inputs and outputs when 
a change is introduced into a system. Theories of practice improvement often are depicted 
visually in a logic model (a graphical representation of causal links between the resources, 
activities, outputs, and outcomes that can be used to guide implementation, evaluation, and 
improvement efforts). 


What does a networked improvement community do? 

The work of a networked improvement community is defined by three primary tasks: develop 
a theory of practice improvement; use continuous improvement research methods (such as 
plan-do-study-act cycles) to implement, test, and redesign an intervention in an iterative 
manner; and build a measurement and analytics infrastructure (Russell et al, in press). 

Develop a theory of practice improvement. The first task involves conducting a root-cause 
analysis, formulating an aim statement, and generating hypotheses about the changes in 
practice that will drive improvement to meet the aim. 

The root-cause analysis by the Michigan Focus Networked Improvement Community, 
which was facilitated by REL Midwest, focused on math achievement gaps in focus schools. 
Participants saw math achievement gaps as a driver of focus school identification and an 
important subject on which to concentrate efforts. In answer to the question “What are 
the root causes of math achievement gaps in Michigan Focus Networked Improvement 
Community schools?” participants developed a list of potential causes, including lack 
of math fluency skills. Participants voted to focus on this cause as the primary driver 
of achievement gaps in math in Michigan Focus Networked Improvement Community 
schools. A fishbone diagram — a tool adapted from the business sector — can be used to 
guide participants in visually representing the problem, its root causes, and the factors 
that contribute to each root cause (Berwick, 2008; Bryk et al, 2015; see appendix A for a 
fishbone diagram template). Participants in the Michigan Focus Networked Improvement 
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Community identified five major “bones” of the fishbone diagram that contribute to the 
problem of low math fluency skills: 

• Time: reteaching math concepts leaves limited time for additional practice. 

• Priority: teachers use breaks in instruction to focus on literacy practice rather than 
math practice. 

• Skill: teachers lack the skills needed to monitor student progress and target 
instruction. 

• Commitment: teacher professional development is most often centered on literacy 
instruction rather than math instruction. 

• Ownership: teachers do not embed math fluency into daily teaching. 


After the root causes are identified, the networked improvement community selects one or 
two root causes on which to focus and develops an aim statement that aligns with the root 
causes selected. The Michigan Focus Networked Improvement Community’s aim state- 
ment was: “Students in the bottom 30 percent of math achievement will achieve mastery 
on grade-level benchmarks in math fluency by the end of the school year.” This goal was 
aspirational but helped organize efforts to achieve it. 

Once the aim is established, the networked improvement community then hypothesizes 
what changes in practice would drive the improvement needed to achieve its aim. Partici- 
pants in the Michigan Focus Networked Improvement Community mapped out perceived 
factors that would influence students’ ability to achieve mastery on grade-level benchmarks 
in math fluency. Participants felt that time, data use for continuous improvement, teachers’ 
skills for differentiating instruction, lack of coherence in curriculum and support struc- 
tures, and family and student engagement were all factors that accounted for achievement 
gaps in math, particularly math fluency. 
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After the Michigan Focus Networked Improvement Community participants developed 
a theory of practice improvement and hypothesized about factors that could affect the 
outcomes specified in the aim statement, they came to a consensus about which factor to 
focus on: time. The group developed an intervention in which students would practice 
their math fluency skills for at least 15 minutes per day. The REL Midwest project team 
developed measurement tools to track these efforts, including teacher logs and principal 
observation protocols to monitor how students and teachers spent those 15 minutes. 


Use continuous improvement research methods. The second task of a networked 
improvement community is to use continuous improvement research methods, such as 
plan-do-study-act cycles, to implement a change in practice based on the theory of practice 
improvement (Bryk et al, 2015). All participants in a networked improvement commu- 
nity may plan to implement the same change or may choose to implement variations of 
the change identified in their respective contexts. Next, the participants implement the 
change and observe the process, collecting formative evidence of its success. In the analy- 
sis stage, participants examine the evidence and determine what tweaks need to be made 
for the change in practice to be more successful. Finally, they implement the modified 
practice and test it again, often at additional sites to learn from variation across these 
sites. The scale of the change is small initially — for example, changing the way in which 
teachers respond to student questions in a particular subject — but then grows in scale as 
the networked improvement community continues its work. Cycles may be as short as two 
weeks or as long as three months. 
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Build a measurement and analytics infrastructure. The final task of a networked 
improvement community is to build a measurement and analytics infrastructure to deter- 
mine whether the change in practice led to improvement. At first, measurement may 
be informal, based on the observations of participants, and focused on process-oriented 
measures (measures related to how well the change has been implemented). But as the 
change in practice is scaled to additional sites, the need arises for a common measure- 
ment system that involves the collection of data across sites, with the goal of being able 
to examine whether the change in practice led to a change in the primary driver and the 
ultimate aim (Bryk et al, 2015). Bryk et al. (2015) emphasize the importance of practi- 
cal measurement (measures that are directly related to the problem identified by the net- 
worked improvement community rather than more traditional accountability measures). 
Measurement should occur often and be embedded within regular organizational routines. 
As participants engage in this process, they develop an understanding of the networked 
improvement community process and how it can be used to scale other initiatives in their 
respective work sites. 

Although educators are the primary participants in the process, a network hub — often 
composed of researchers — facilitates the process (Bryk et al., 2015). The network hub pro- 
vides expertise on continuous improvement research, guiding the participants through 
each step of the process. The network hub also monitors the process of continuous improve- 
ment research across sites and provides feedback to participants about what is working well 
and where there may be challenges. Finally, the network hub plays the practical role of 
convening participants. As Bryk et al. (2015, p. 159) conclude, “tending to the needs of 
the community is foundational for everything else.” The REL Midwest project team served 
the role of network hub for the networked improvement communities in Michigan and 
Minnesota. 
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How Regional Educational Laboratory Midwest facilitated 
networked improvement communities in Michigan and Minnesota 


The U.S. Department of Education granted flexibility from elements of the Elementary 
and Secondary Education Act to Michigan in 2011 and to Minnesota in 2012. Both states 
subsequently revised their statewide systems of support to promote school improvement for 
priority schools and focus schools. In 2015 state education agency leaders from Michigan 
and Minnesota participating in the REL Midwest School Turnaround Research Alliance 1 
requested assistance with establishing and sustaining networked improvement communi- 
ties to advance their efforts to raise student achievement and narrow achievement gaps 
in priority and focus schools. The REL Midwest project team relied on written guidance 
from the Carnegie Foundation for the Advancement of Teaching to help fulfill this request 
(Bryk, Gomez, & Grunow, 2011; Bryk et al., 2015). 

The project team’s assistance addressed both short-term and longer-term needs. In the 
short term the project team would play a strong facilitative role to assist each state in estab- 
lishing networked improvement communities and in building the knowledge and skills 
of participants to engage in and lead this type of collaborative research partnership. The 
development of tools and resources would address longer-term needs by supporting partici- 
pants as they took over facilitative responsibilities after REL Midwest’s participation in the 
project ended. 
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Michigan Focus Networked Improvement Community 


At the request of the Michigan Department of Education, the project team established 
the Michigan Focus Networked Improvement Community to support school improvement 
in select focus schools. The Michigan Department of Education had an evidence-based 
system of supports in place for priority schools founded on turnaround models from prior 
years. However, focus schools had only recently been identified through the state account- 
ability system, and as a consequence, the system of support needed was less clear. Forming 
a networked improvement community offered the Michigan Department of Education an 
opportunity to test improvement strategies in local settings in ways that were relevant 
and useful to practitioners in those schools. The goal of the Michigan Focus Networked 
Improvement Community is to address disparities in student achievement within schools 
by engaging personnel at the school, district, intermediate school district, and state levels 
in iterative plan-do-study-act cycles. Participants include state education agency repre- 
sentatives, intermediate school district administrators, district representatives, and focus 
school principals (figure 1). The Michigan Focus Networked Improvement Community 
works across five focus schools in two districts. The two districts are each part of an inter- 
mediate school district. 

Through monthly meetings, participants in the Michigan Focus Networked Improvement 
Community identified inequality in math achievement as their primary concern and 
hypothesized that math fluency skills were a primary driver of this inequality. They devel- 
oped an intervention to address math fluency skills for students in the bottom 30 percent 


Figure 1. Organization of the Michigan Focus Networked Improvement Community 



The goal of the 
Michigan Focus 
Networked 
Improvement 
Community is to 
address disparities 
in student 
achievement 
within schools 
by engaging 
personnel at the 
school, district, 
intermediate 
school district, 
and state levels in 
iterative plan-do- 
study-act cycles 


Note: The Regional Educational Laboratory Midwest project team supported the Michigan Focus Networked 
Improvement Community but operates outside the governance structure of Michigan public schools. The proj- 
ect team worked primarily with the Michigan Department of Education to establish the networked improvement 
community and therefore is placed next to Michigan Department of Education staff as a joint partner but not 
connected to the network under a formal arrangement. 

Source: Authors. 
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of math achievement and determined which formative and summative indicators would be 
used to measure changes in math fluency. 


Minnesota Statewide System of Support Networked Improvement Community 


The Minnesota Department of Education approached REL Midwest for assistance with 
improving the supports (such as professional development and coaching) that it provides to 
its six Regional Centers of Excellence, which provide direct support for school improvement 
to priority and focus schools across the state. The REL Midwest project team, in conjunction 
with the director of school support at the Minnesota Department of Education, decided that 
forming the Minnesota Statewide System of Support Networked Improvement Community 
would be a promising approach to studying and improving the supports that the Minnesota 
Department of Education provides to the regional centers. The directors of the regional 
centers and their staff draw on improvement science methods by engaging school leadership 
teams in needs assessments and continuous data review to define priorities and implement 
evidence-based practices that address these priorities, but they had not used improvement 
science methods in a networked improvement community. The networked improvement 
community builds on staff members’ existing expertise in improvement science and allows 
for learning across the six regional centers. REL Midwest facilitates the networked improve- 
ment community in partnership with the Minnesota Department of Education’s director 
of school support, who oversees the Cross-agency Implementation Team, which oversees 
implementation of Minnesota’s statewide system of support. 

The Minnesota Department of Education decided that the members of the Cross-agency 
Implementation Team — which include department staff (the director of school support, 
the program manager of the Regional Centers of Excellence, and content-area specialists) 
and the directors of the regional centers — would participate in the Minnesota Statewide 
System of Support Networked Improvement Community (figure 2). Through an iterative 
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Figure 2. Organization of the Minnesota Statewide System of Support Networked 
Improvement Community 



Note: The Regional Educational Laboratory Midwest project team supported the Minnesota Statewide System 
of Support Networked Improvement Community but operates outside the governance structure of the Minne- 
sota Cross-agency Implementation Team. The project team worked primarily with the Minnesota Department of 
Education to establish the networked improvement community and therefore is placed next to Minnesota De- 
partment of Education staff as a joint partner but not connected to the network under a formal arrangement. 

Source: Authors. 
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process, the networked improvement community identified challenges that the Minne- 
sota Department of Education faces in supporting the regional centers, including clearly 
communicating expectations for closing achievement gaps. Through a root'Cause analy- 
sis, participants identified several factors that contribute to the problem, including lack of 
knowledge about evidence-based practices related to closing achievement gaps; lack of pro- 
fessional development opportunities focused on closing achievement gaps; and insufficient 
communication among the Minnesota Department of Education, the regional centers, and 
the staff who work in focus schools to close achievement gaps. 

What the project found 


This section reports on five key lessons learned during implementation of networked improve- 
ment communities in Michigan and Minnesota. In doing so, this report aims to contribute to 
a nascent literature on considerations that emerge in the process of establishing and sustain- 
ing networked improvement communities. The lessons learned were distilled from a series 
of conversations and brainstorming sessions held by the REL Midwest project team for each 
networked improvement community. Over a period of several months, the REL Midwest 
project team met regularly with colleagues with expertise in continuous improvement 
research to discuss the progress of each project and any challenges encountered. These con- 
versations were informed by feedback from networked improvement community participants 
provided during project debrief conversations with the REL Midwest project team at critical 
junctures in the project, such as after networked improvement community meetings. These 
conversations provided opportunities throughout the project to identify common challenges 
and lessons learned across each networked improvement community. 

Build a cohesive team with members representing different types of expertise 

While REL Midwest served as the facilitator for each networked improvement communi- 
ty, it was critical to do so in partnership with a champion at the state education agency. 
Champions are usually decisionmakers in the organization and have the power to commit 
institutional resources to the project. Champions help recruit participants, contextualize 
the work for participants, and advocate for the process both within the state education 
agency and across the stakeholder groups represented in each networked improvement 
community. Champions are most effective when they are viewed as knowledgeable and 
valuable by other stakeholders. 

Networked improvement communities also require distinct types of expertise — including 
content, context, and research expertise — to ensure accuracy and build legitimacy for their 
work. Content experts specialize in the content or disciplinary areas that the statewide 
system of support addresses. Context experts deeply understand the organization of their 
state’s education system, the responsibilities of stakeholders in the system, how those stake- 
holders interact with each other, and how those stakeholders are both supported and chal- 
lenged. They are knowledgeable about the political and personal landscape of the local 
context. And they connect the networked improvement community to resources, antici- 
pate and propose solutions to barriers to implementation of an intervention, and provide 
guidance on how to structure the networked improvement community for sustainability. 
Finally, research expertise supports the data collection and analysis work of networked 
improvement communities and contributes thinking to the development and assessment 
of outcome measures. 


Networked 
improvement 
communities 
require distinct 
types of expertise- 
including content, 
context, and 
research expertise 
-to ensure 
accuracy and 
build legitimacy 
for their work 
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Because the Michigan Focus Networked Improvement Community is trying to address 
students’ mastery of benchmarks for math fluency, the REL Midwest project team conskh 
ered it important to have a math content expert who could speak knowledgeably about 
key considerations for student proficiency at various grade levels, the availability of assess- 
ments, and proposed curricula for the intervention. In addition, the multilayered system 
of support in Michigan required at least one representative from each governance level as 
context experts to provide insight into the problem and to help address barriers to develop- 
ing the intervention, such as those related to how participating schools and districts make 
decisions about math curricula and teaching practices. Finally, data specialists at the dis- 
trict level were included to provide research expertise. This expertise was supplemented by 
the project team, which supported the data collection and analysis work of the networked 
improvement community and contributed to the development and assessment of outcome 
measures. 

Recruitment for the Minnesota Statewide System of Support Networked Improvement 
Community was straightforward. The participants were already members of the Cross- 
agency Implementation Team, led by the Minnesota Department of Education (the cham- 
pion). The participants include the directors of the six Regional Centers of Excellence, 
who serve as content experts. The directors previously held positions as teachers and 
school leaders and therefore understand how schools work and the challenges faced by 
schools. They also are intimately familiar with the context of schooling and the broader 
needs of their particular regions. The Minnesota Department of Education’s director of 
school support and its manager of the Regional Centers of Excellence understand the 
state-level context, including the goals of the statewide system of support and the details of 
how it functions across the state. Finally, the Cross-agency Implementation Team includes 
specialists from the Minnesota Department of Education who are experts in program 
implementation and data collection and analysis. In sum, participants in the networked 
improvement community understand the challenges of the problem, have the capacity 
to implement the changes designed from the process, and can collect outcome data to 
measure success. The REL Midwest project team worked in collaboration with the Minne- 
sota Department of Education data specialists to provide research expertise. 


In both states some 
participants came 
to the process with 
preconceptions 
about what 
a networked 
improvement 
community is and 
does based on their 
past participation 
in communities 
of practice and 
professional 
learning 
communities 


Reduce uncertainty by clarifying what participation entails 

After potential participants were identified, the REL Midwest project team met with them 
to explain the networked improvement community’s goals and intended outcomes. In 
Michigan these early conversations illuminated a need to provide information about what 
a networked improvement community is and how it differed from but built on continuous 
improvement work that participants already were undertaking, about what participation in 
a networked improvement community involves, and about expectations for participating. 
In both states some participants came to the process with preconceptions about what a 
networked improvement community is and does based on their past participation in com- 
munities of practice and professional learning communities. Communities of practice are 
groups of people who share a common purpose and learn together to improve practice 
(Wenger-Trayner & Wenger-Trayner, 2015). Although networked improvement commu- 
nities are similar to communities of practice in that members share a common purpose 
and learn collaboratively, they differ in that networked improvement communities use a 
structured process to identify a problem to address, decide on and implement an interven- 
tion, and measure the outcome of the intervention. Similarly, networked improvement 
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communities share some features of professional learning communities — specifically, col- 
laborative learning to develop a shared understanding of a common topic (Hord, 2009) — 
but focus on the impact of a common intervention introduced across sites rather than on 
improving individual practice. Clarifying the differences between a networked improve- 
ment community and other collaborative processes provided a common understanding of 
participants’ responsibilities and how each participant’s expertise would be used to support 
the goals of the networked improvement community. 


The REL Midwest project team intentionally structured recruitment activities to build 
commitment among participants and clarify participant responsibilities, roles, and expec- 
tations. In Michigan the Michigan Department of Education partner made initial contact 
with intermediate school districts. The project team then traveled to the Michigan 
Department of Education to meet with the recruited intermediate school districts at a 
state-led meeting for focus school support providers. At this meeting, the idea of a net- 
worked improvement community was introduced and details were given on how it would 
be carried out. The project team then worked with intermediate school district officials to 
identify schools and districts that could also participate. This multistep process of recruit- 
ment at each governance level — first state, then intermediate school district, then dis- 
tricts, then schools — was crucial to building relationships with partners across the state, 
intermediate school district, district, and school levels, as well as ensuring that the right 
stakeholders would be involved in the process. 


In both states 
the multiphase 
recruitment 
process was 
important for 
building trust 
among participants 


In Minnesota the project team held an initial meeting with the Minnesota Department of 
Education to determine who would participate in the networked improvement communi- 
ty. At the meeting it was agreed that members of the Cross-agency Implementation Team 
would participate. During a second meeting the project team and the Minnesota Depart- 
ment of Education determined the specifics of the project plan and set the agenda for the 
project kickoff meeting with the Cross-agency Implementation Team. The project team 
vetted the materials that would be provided to participants at that meeting with the Min- 
nesota Department of Education. Prior to, during, and after the kickoff of the networked 
improvement community, the project team described the networked improvement commu- 
nity process and answered questions about participation. During initial meetings of both 
networked improvement communities, the project team provided participants with a brief 
handout that described the proposed networked improvement community and the respon- 
sibilities of each participant (see box 2 and appendix B for an example from Minnesota). 


In both states the multiphase recruitment process was important for building trust among 
participants. Engaging in the work of a networked improvement community requires par- 
ticipants to take a close look at their practices, be willing to admit that some practices 
are not working well, and make changes accordingly. This may be challenging for some 
because the process requires them to accept critical feedback on work that they were 
deeply invested in. To facilitate this process the project team had to build trust with the 
participants as well as among the participants themselves. The multiphase recruitment 
process helped do so by providing numerous opportunities for the project team to interact 
with participants prior to the networked improvement community’s work. Although this 
takes time, it is important to build the foundation for the networked improvement com- 
munity’s success. Time demands on participants can be minimized by aligning recruitment 
meetings with existing meetings. In Minnesota all recruitment meetings occurred in con- 
junction with standing Cross-agency Implementation Team meetings. 
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Box 2. Responsibilities as a networked improvement community participant 


The Regional Educational Laboratory Midwest project team developed a list of frequently asked 
questions for each networked improvement community to clarify the responsibilities of partic- 
ipants. What follows is an excerpt from the document used in Minnesota (see appendix B for 
the complete document). 

As a participant in this NIC, you will be asked to: 

• Participate in regular meetings to identify specific problems. 

• Assist in developing a solution to each problem. 

• Provide information about how the change has been implemented. 

• Participate in discussions to refine the change as it is implemented. 


Build engagement by aligning work with ongoing efforts 

In Michigan, participants wanted to know how the networked improvement community’s 
work differed from the continuous improvement work that they were already engaged in, 
which included annual needs assessments. The REL Midwest project team explained the 
value of the networked improvement community approach, which involves rapid, iterative 
cycles that use formative, rather than summative, data. The project team further explained 
the value of leveraging a network of schools and districts for learning how change varies 
by context. The project team worked with state education agency staff in Michigan to use 
terminology and align networked improvement community work to the state’s blueprint 
for its statewide system of support. In presentations the project team explicitly demonstrat- 
ed how improvement science methods could be used to achieve the aims of the statewide 
system of support. For example, Michigan’s school turnaround plans suggest that districts 
create a protocol for reviewing multiple sources of data; the project team discussed how the 
networked improvement community model could be used as one such protocol for data 
review. 


Networked 
improvement 
communities focus 
on problems that 
are important 
and actionable 


In Minnesota the project team mapped the networked improvement community’s work 
to the comprehensive statewide system of support evaluation plan to illustrate how the 
networked improvement community’s activities would address components of the plan. 
The kickoff meeting for participants was led by the state education agency partner, who 
explained how the project originated within the state education agency and complement' 
ed existing efforts to evaluate and improve the supports provided to Regional Centers of 
Excellence staff using continuous improvement methods. By addressing participants’ con- 
cerns about duplication of effort and by describing the value of the networked improve- 
ment community, the project team was able to establish trust and engage participants in 
the networked improvement community’s work. 

Use tools and resources from improvement science to identify a problem that is important and 
specific enough to be able to act on 

Networked improvement communities focus on problems that are important and action- 
able, such as failure rates in developmental math in community college or teachers’ abili- 
ties to implement a specific pedagogical practice (Bryk et al, 2015). Identifying important 
problems can enable initial and sustained institutional support, buy-in and ownership 
of participants, and diffusion of practices and processes that are produced through the 
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networked improvement community’s work. However, problems also need to be within 
the participants’ control if they are to enact changes in practice through rapid plan-do- 
study-act cycles and learn from those changes. In initiating the Michigan and Minnesota 
networked improvement communities the REL Midwest project team worked with partic- 
ipants to identify problems that were both important and actionable. In both states the 
focus of the networked improvement community is the implementation of the statewide 
system of support. However, to make progress on improving supports to schools served by 
the statewide system of support, it was necessary for each networked improvement com- 
munity to engage in an iterative process to further narrow the problem on which it would 
focus. This process involved multiple conversations with key stakeholders. 


One strategy for narrowing the scope of work to a specific problem is to conduct a root- 
cause analysis (Bryk et al., 2015). The project team adapted this suggestion from the litera- 
ture by developing two activities (see box 3 and appendix A). Activity 1, creating a focused 
problem statement, enabled participants to move from specific problems encountered in 
their daily practice during the previous day, week, month, and year to a focused problem 
statement that would guide the networked improvement community’s work. Using partic- 
ipants’ daily experiences rooted the problem in practice and helped narrow the scope to a 
problem that was specific and actionable. This also helped participants connect the net- 
worked improvement community’s work to their current responsibilities, increasing buy-in. 
The project team then led participants through an exercise using a fishbone diagram to 
identify the root causes of the problem statement. During the course of three months, 
using this set of structured activities, Michigan Focus Networked Improvement Communi- 
ty participants narrowed their focus to the specific problem of the lack of time for students 
to practice their math fluency skills on a daily basis. 


Problems need 
to be within the 
participants’ 
control if they are 
to enact changes 
in practice through 
rapid plan-do- 
study-act cycles 
and learn from 
those changes 


Box 3. Root-cause analysis: Overview of activities 1 and 2 

Each root-cause analysis activity requires approximately 1.5 hours. Activity 1 helps individu- 
als create a focused problem statement based on problems they have encountered in their 
daily practice. Activity 2 uses a fishbone diagram to identify root causes. The primary problem 
statement becomes the “head” of the fishbone diagram. The focused problem statement com- 
prises the key factors that make up each major “bone” (Berwick, 2008; Bryk et al., 2015). See 
appendix A for more detailed instructions about sample activities for the root-cause analysis. 

Activity 1. Participants are encouraged to brainstorm problems that they have encountered in 
the past day, week, month, and year. Participants document these problems on sticky notes, 
which are collected by facilitators and placed on a large poster board. Participants take turns 
sharing the problems with the group and then work together to categorize them into broad 
buckets, such as “leadership” or “alignment.” Participants draft a problem statement for each 
bucket and consolidate the problem statements into one focused problem statement to guide 
the networked improvement community’s work. 

Activity 2. Participants work in small groups to identify root causes of each problem identified 
in activity 1. Using a fishbone diagram, participants visually represent the problem and the 
contributing factors to each (Berwick, 2008; Bryk et al., 2015). They ask “Why?” until they 
have identified several contributing factors — or root causes — of each problem. After the group 
decides on a final depiction of the problems and root causes on the fishbone diagram, the 
facilitator leads a whole-group discussion to build consensus on which of the root causes the 
group will focus. 
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Embed capacity building to develop additional expertise for using continuous improvement research 
to address problems of practice 


Each participant brings important knowledge and skills to the networked improvement 
community; however, participants may not have experience in designing and engaging in 
continuous improvement research within a networked improvement community. In both 
states there was a desire to build capacity for continuous improvement using the networked 
improvement community model. An important goal of these projects — beyond addressing 
the identified problems of practice — was to develop shared expertise to support lasting uses 
of continuous improvement research, including building participants’ skills to be able to pur- 
posefully plan and measure changes in practice. To meet this goal, the REL Midwest project 
team intentionally structured these projects so that the networked improvement communi- 
ties would engage in one or two plan-do-study-act cycles with project team facilitation and 
then continue to engage in additional plan-do-study-act cycles on their own as needed. 

Conclusion 


Although there is practical guidance for how networked improvement communities should 
structure continuous improvement work, few published accounts describe the process of 
forming a networked improvement community. This report shares five lessons learned by 
REL Midwest while forming networked improvement communities in Michigan and Min- 
nesota. As researchers and state education agency and school district leaders set out to form 
networked improvement communities, they can draw on these lessons to guide their efforts. 


An important 
goal-beyond 
addressing the 
identified problems 
of practice-was 
to develop shared 
expertise to 
support lasting 
uses of continuous 
improvement 
research 


These lessons highlight the importance of relationships during the process of establishing 
and facilitating networked improvement communities. Continuous improvement research 
relies on the ability of networked improvement community participants to work together. 
Facilitators cannot foresee and account for all barriers that may arise. For example, in Min- 
nesota the networked improvement community’s work required the Minnesota Department 
of Education and Regional Centers of Excellence staff to look closely at their practices — a 
task that was challenging because it required them to accept critical feedback on work that 
they were deeply invested in. Participants engaged in many difficult but important conver- 
sations. As a result, they had to reset their expectations about the timeline for the project. 
The REL Midwest project team has shared these lessons so others may anticipate potential 
challenges as they set out to establish networked improvement communities. 


This report has three limitations. First, this report examines only two instances of initi- 
ating and sustaining a networked improvement community. Other networked improve- 
ment communities might experience other types of barriers that are not described in this 
report, such as a major change in senior leadership over the course of the project. Second, 
the REL Midwest project team set out not to systematically study networked improve- 
ment communities and their impact, but to report on experiences during the process of 
implementing guidance found in the literature. It is unknown whether the investment 
in these networked improvement communities will have an impact on the outcomes of 
interest. Finally, the project team’s work in Michigan and Minnesota focused on building 
each state’s capacity to engage in continuous improvement research — including additional 
plan-do-study-act cycles — after REL Midwest was no longer involved. It is not yet known 
whether the states will be able to sustain these efforts without REL Midwest’s support or to 
generalize what they learned to other content areas. 
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Appendix A. Sample activities to define 
the problem and identify root causes 


This appendix provides a set of sample activities that a facilitator can use to bring a group 
to consensus about the definition of a problem and its root causes. Activity 1 helps individ- 
uals create a focused problem statement and categorizes potential causes. Activity 2 uses 
the fishbone diagram to identify root causes. The primary problem statement becomes the 
“head” of the fishbone diagram. The focused problem statement comprises the key factors 
that make up each major “bone.” 

Together, the activities should take approximately 2 hours and 15 minutes, depending on 
the size of the group. 

Activity 1: Create a focused problem statement 

Goal: Bring the group to a consensus on the specific problem statement. 

Strategy: Start with small, specific problems, and then combine issues to form a general 
statement. 

Materials: Chart paper, markers, sticky notes, pens/pencils. 

Step 1: Brainstorm problems (20 minutes) 

Encourage participants to think of a problem that they have experienced in the past day 
related to the topic at hand (such as providing supports to low-performing schools). An 
example might be, “Earlier, I tried to conduct a site visit, but the school schedule had 
changed and I could not meet with the assistant principal as planned.” Have them write 
the problems down on individual sticky notes. Only give them a few minutes to do so. 

Next, encourage participants to think of one to three problems that they have experi- 
enced in the past week related to the topic at hand. Do the same for the past month and 
year. Each problem should be written on a sticky note, with an indication of whether the 
problem arose in the past day, week, month, or year. 

Step 2: Share and group problems (15 minutes) 

Solicit a few volunteers to read their problems aloud. 

Have participants get together in small groups and group all the problems written on the 
sticky notes together into general categories (such as “scheduling” or “lack of knowledge of 
best strategies”). 

If participants note that a problem could relate to more than one general category, encour- 
age them to attach each problem to its dominant category — the one that characterizes it 
most centrally. 
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Step 3: Write a problem statement (15 minutes) 


Assign a few participants to each general category of problems. Have the participants work 
together to draft a focused problem statement for each category. Encourage the partici- 
pants to be as specific as possible. 

Step 4: Build consensus (25 minutes) 

Read the drafted problem statements aloud and ask if there are any revisions. Ask partic- 
ipants whether the drafted statements are specific enough. Make revisions as a group and 
finalize the statements. 

Use the individual problem statements to draft one large, focused problem statement as a 
group. 

Activity 2: Root-cause analysis 

Goal: Determine the root causes of the problems defined in activity 1. A root cause is the 
reason why the problem occurs, unearthed in this exercise after several iterations of asking 
why a particular phenomenon exists. 

Strategy: Assign groups of participants to each individual problem, now a subproblem under 
the general problem statement, developed in activity 1 and have them work through the 
fishbone diagram (figure Al). 

Materials: Projector, sticky notes, pens/pencils, dot stickers. 

Step 1: Work in small groups on subproblems (20 minutes) 

Project the fishbone diagram, if possible, and write out the general problem statement 
developed during activity 1 at the head. Each “bone” should be one of the subproblems 
identified in activity 1. Each subproblem should have multiple root causes (these are the 
sticky notes from step 1 of activity 1). 

Ask participants whether other subproblems should be listed. There should be only four or 
five. These subproblems now should be thought of as causes. If a subproblem does not make 
sense as a cause of the problem statement, keep that sticky note in a “parking lot”; perhaps 
it is actually a root cause or perhaps it just illustrates a root cause. 

Divide participants into groups. Each group should have one subproblem, depending on 
the size of the group. Each group should get a copy of the fishbone diagram. 

Encourage participants to use the sticky notes they grouped in activity 1 to think about 
the subproblem. Each group should ask, “Why does this happen?” for the subproblem, and 
fill out the fishbone diagram with answers until they get to a root cause. A good rule of 
thumb is five “why” questions, but that is not a hard-and-fast rule. Each group should focus 
only on the causes of their particular subproblem. 

Remind participants to focus on causes and not solutions. 
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Step 2: Present the root causes (25 minutes) 


Have each group present its root causes to the larger group. Ask the larger group if it has 
anything to add. Make revisions as needed. 

As participants read out their fishbone diagram, add to the larger fishbone diagram that 
is projected on the wall so participants can see how they are building the whole diagram 
together. Another option is to have groups that finish early do this while they are waiting 
for others. 

Step 3: Engage participants in a discussion of the root causes (15 minutes) 

Did the process yield the results they expected? What was surprising? What was not 
surprising? 

After participants have shared their reflections on the results of the root-cause analysis, ask 
them to think about which of the root causes they want to address first. 

Provide each participant with three dot stickers. Ask each participant to vote on which 
root cause they want to address by placing their dots next to the root cause they want to 
address first. Participants can distribute the dots in any way. The cause identified as the 
highest priority will be addressed first by the networked improvement community. 


Figure Al. Fishbone diagram 
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Appendix B. Frequently asked questions 
about networked improvement communities 


This appendix contains the frequently asked questions handout created by the Region- 
al Educational Laboratory Midwest project team for the Minnesota Statewide System of 
Support Networked Improvement Community. This handout was used to support the 
recruitment of networked improvement community participants. 


REL 

MIDWEST 

Regional Educational Laboratory 

at American Institutes for Research 


Developing a Networked Improvement Community 
to Support Regional Centers of Excellence 

Frequently Asked Questions 


[ 4 * 


What is a networked improvement community (NIC)? 


A networked improvement community (NIC) is a group of individuals who come together 
across diverse contexts to work on a problem of practice. It is a scientific learning 
community in which quick, actionable research is used to make changes to practice in 
complex organizations. These changes are continually measured and refined to address a 
specific problem of practice. NICs are different from communities of practice because the 
goal is to not only share across contexts but also work together to invent and continually 
refine solutions to problems of practice. 


O Why develop a networked improvement community to support 
Regional Centers of Excellence? 


The Regional Centers of Excellence are relatively new in Minnesota, and the Minnesota 



Department of Education (MDE) would like to better understand how well it is supporting 


these Regional Centers of Excellence as they implement the statewide system of support. 


Through an iterative process, the NIC will leverage the expertise of staff from MDE, the 


Regional Centers of Excellence, and Regional Educational Laboratory (REL) Midwest to 


■ Identify specific areas of need (Plan). 

■ Intervene to improve supports to address those needs (Do). 

■ Measure any changes that occur (Study). 


■ Refine the intervention (Act). 


X 

gf p |g Why have I been chosen to participate in this continuous improvement process? 


You have been chosen to participate in this process because you work with MDE or the Regional Centers 
of Excellence. Participation is voluntary and should not be in addition to work that you are doing; it should 
be complementary. 

Tie Plan/Do/Study/Act graphic is from Langley, G. L. r Moen, R., Nolan, K. M, Nolan,!. W., Norman, G L., & Provost, L. R (2009). The improvement guide: A practical approach to enhancing organizational performance (2nd ed.). 
San Francisco, CA: Jossey-Bass. 
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What are my responsibilities as a participant in the NIC? 




As a participant in this NIC, you will be asked to 

■ Participate in regular meetings to identify specific problems. 

■ Assist in developing a solution to each problem. 

■ Provide information about how the change has been implemented. 

■ Participate in discussions to refine the change as it is implemented. 

During fall 2015, you may be asked to complete an interview or survey about your experiences with the 
statewide system of support rollout. Your responses will provide data that will help in the planning of the 
continuous improvement process. 


# 0) What is REL Midwest and what is its role in the continuous improvement process? 


REL Midwest is part of a network of 10 regional 
educational laboratories funded by the U.S. 

Department of Education’s Institute of Education 
Sciences. Each REL serves a designated region 
of the United States and focuses on the national 
priority of helping states and districts use data 
and analysis to address important policy and 
practice issues, with the goal of improving 
student outcomes. 

REL Midwest will support the NIC by providing tools to identify a problem of practice that is directly related to 
the statewide system of support. REL research staff also will develop indicators and instruments to specify 
how changes to practice that aim to improve this problem will be measured. 





Contact Information 


For more information, please contact REL Midwest at American Institutes for Research: 

REL Midwest 

Attn: Amy Proger or Victoria Cirks 
American Institutes for Research 
1120 East Diehl Road, Suite 200 
Naperville, IL 60563 
866-730-6735 | RELMidwest@air.org 

3942b_10/15 
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Note 


1. The REL Midwest School Turnaround Research Alliance is a partnership between 
researchers at REL Midwest and educators in REL Midwest Region states. The alliance 
builds capacity among members to identify and address school turnaround challenges 
related to policy and practice through regional research, technical assistance, and dis- 
semination projects. 
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